In recent years, in-vehicle cameras have been spreading for the purpose of supporting a driver's driving. For example, there is a system that performs viewpoint conversion on each image of a plurality of cameras mounted around a vehicle, composes the image, and presents an overhead view image of the surroundings of the vehicle to a driver, or a system that controls a vehicle by using a result of measuring a position of an object outside the vehicle.
The accuracy of the in-vehicle camera is determined by camera-specific parameters (internal parameters) such as optical characteristics, such as a focal length or a lens distortion, or a size of an imaging device, and parameters (external parameters) related to a mounting position or an angle (a roll angle, a pitch angle, or a yaw angel) of each camera. The internal parameters and the external parameters are collectively referred to as camera parameters.
Although the camera is attached to the vehicle at a position or an angle conforming to a predetermined design value, an error occurs at this time and the accuracy of the camera deteriorates. Therefore, a mounting error from a design value of a camera parameter has been corrected. This is referred to as calibration or correction.
Since the calibration is required to correct an error with high accuracy, there is no method of mechanically adjusting an installation state of a camera, and a current installation state is estimated from a photographed image. As a method of estimating an installation state from a photographed image, a method is generally adopted which accurately places and photographs a calibration mark (also referred to as a pattern or a calibration chart) printed on a paper or a plate at a predetermined position and correcting a camera parameter such that an actual photographed image matches an image photographed by a camera manufactured and installed according to a design value.
Meanwhile, such a calibration has been generally performed before shipment of a vehicle at a place where a camera is installed, for example, in a factory, but recently, as disclosed in PTL 1 or PTL 2, it has been proposed to execute calibration even after factory shipment by using an object such as a white line, for example, as a calibration mark during traveling. The calibration during traveling is excellent in that it is also possible to cope with a change in a camera parameter that can occur after factory shipment.
As one of the causes of the change in the camera parameter afterwards, it is considered that a loading state of an object with respect to a vehicle changes according to circumstances, for example, a difference in the number of passengers, a difference in places to sit, and a difference in a method of loading a baggage. When such a loading state changes, the attitude of the vehicle changes and the installation state of the camera with respect to the ground (that is, camera parameter) changes.